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PREFACE 


This  document  was  prepared  by  the  Institute  for  Defense  Analyses  for  the 
Advanced  Research  Projects  Agency  (ARPA),  under  the  ARPA  Assignment,  Synthetic 
Theatre  of  War  (STOW),  and  fulfills  the  objective  of  providing  “technical  support  focused 
to  the  needs  of  STOW.  Areas  to  be  addressed  include . . .  technology  assessments  relevant 
to  the  sucessful  execution  of  STOW  (e.g.,  pertaining  to  scaleability  . . . ).” 

The  document  was  reviewed  by  Dr.  Richard  J.  Ivanetich,  Director  of  the  Computer 
and  Software  Engineering  Division,  Institute  for  Defense  Analyses. 
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INTRODUCTION  AND  SUMMARY 


Introduction 

At  the  request  of  Colonel  Robert  Reddy,  Assistant  Director  of  the  Advanced 
Research  Projects  Agency/ Advanced  Systems  Technology  Office  (ARPA/ASTO)  and  Pro¬ 
gram  Manager  for  Synthetic  Theater  of  War  (STOW)  as  well  as  Advanced  Distributed  Sim¬ 
ulation  (ADS),  a  Peer  Review  Panel  was  convened  to  review  the  work  being  done  on 
network  scaleability.  The  term  scaleability,  as  used  in  this  document,  refers  to  techniques 
to  maximize  the  number  of  distributed  interactive  simulation  (DIS)  entities  on  a  network 
by  minimizing  the  load  presented  to  the  network,  both  in  terms  of  bandwidth  and  packets 
per  second. 

The  Panel  was  asked  to  consider  the  following  questions; 

•  Are  these  approaches  the  most  promising?  If  not,  what  would  be  better  ? 

•  Do  the  separate  programs  fit  together  to  form  a  coherent  approach? 

•  Are  the  programs  practicing  “good  science”?  Are  good  experimental  methodol¬ 
ogies  being  used? 

•  Are  the  programs  incorporating  leading  edge  technology? 

•  What  is  the  Panel’s  risk  assessment  of  these  approaches,  and  what  can  be  done 
to  manage  risk? 

The  Panel  met  at  the  Institute  for  Defense  Analyses,  Alexandria,  \firginia,  on 
August  19  and  20, 1993.  The  Panel  members  were  as  follows: 

•  Dr.  David  R.  Cheriton,  Stanford  University,  Palo  Alto,  C?alifomia 

•  Dr.  Dale  B.  Henderson,  Los  Alamos  National  Laboratory,  Los  Alamos,  New 
Mexico 

•  Dr.  Duncan  C.  Miller,  Massachusetts  Institute  of  Technology  Lincoln  Laborato¬ 
ry,  Lexington,  Massachusetts 
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Dr.  David  L.  Mills,  University  of  Delaware,  Newark,  Delaware 

Mr.  Stuart  D.  Cheshire,  Adjunct  Panel  Member,  Stanford  University,  Palo  Alto, 
California 


The  Navy  Research,  Development,  Test  and  Evaluation  Division  (NRaD)  of  the 
Naval  Command,  Control,  and  Ocean  Surveillance  Center,  San  Diego,  California,  provided 
a  “read-ahead”  package.  i 

Presentations  were  given  by  the  following; 

•  CDR  Dennis  McBride,  ARPA/ASTO  Program  Manager  for  Scaleability 

•  Ms.  Linn  Hynn,  ARPA/ASTO  Program  Manager  for  the  Distributed  Simulation  { 

Internet 

•  Mr.  Dan  van  Hook,  Loral  Advanced  Distributed  Simulation  (LADS) 

•  Mr.  William  Miller,  American  Telephone  and  Telegraph  (AT&T)  ^ 

•  Dr.  Joshua  Seeger  and  Mr.  Lou  Berger,  Bolt  Beranck  and  Newman  (BBN) 


Summary 

Conclusions  I 

The  panel  provided  the  following  overall  conclusions: 

•  10,000  entities  on  a  secure  network  in  1994  seems  impossible,  given  the  restric¬ 
tions  of  the  Motorola  Network  Encryption  System  (NES). 

i 

•  T- 1  tail  circuits  (about  1 .5  megabits  per  second  throughput)  will  require  approx¬ 
imately  a  90%  traffic  reduction  from  the  full  exercise  load  that  might  be  present¬ 
ed  to  the  backbone.  If  a  T3  (about  45  megabits  per  second)  backbone  is 

available,  then  10,000  entities  should  not  present  a  problem  to  the  backbone.  ^ 

Therefore,  a  90%  reduction  in  traffic  down  the  tail  circuit  is  required. 

•  MCGs  (multicast  groups)  seem  absolutely  required  for  large  numbers  of  entities 
on  the  wide  area  network,  in  conjunction  with  other  reduction  techniques.  Much 

more  study  is  needed  on  the  effects  of  MCGs  versus  traffic  reduction.  If  a  secure  I 

network  is  essential,  then  STOW  needs  to  prepare  to  fall  back  to  less  then 
10,000  entities  for  1994. 

•  Achieving  a  90%  traffic  reduction  >vill  require  at  least  100  MCGs  and  perhaps 

many  more.  If  an  upgraded  NES  vdll  not  do  more  than  16  MCGs  each,  then  ^ 
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there  may  be  a  possibility  to  trade  off  aggregating  traffic  to  provide  increased 
shared  bandwidth  versus  using  parallel  NES  to  obtain  an  increased  number  of 
MCGs.  If  about  6  to  10  NES  are  needed  to  get  100  MCGs,  then  130  NESs  are 
needed  to  support  13  sites.  These  130  NESs  would  cost  about  $4  million. 

•  The  Panel  emphasized  the  importance  of  a  judicious  scenario  setup  to  match  the 
virtual  world  with  the  physical  (network)  sites  in  order  to  achieve  traffic  reduc¬ 
tion.  This  must  always  be  considered,  and  the  interactions  and  logistics  are  com¬ 
plex.  Pathological  conditions  that  would  require  broadcasting  all  traffic  to  all 
sites  must  be  avoided.  Any  possible,  arbitrary  scenario  is  not  feasible.  Knowl¬ 
edge  of  the  intricate  details  of  DSI  configuration  is  vital  to  make  the  most  effi¬ 
cient  utilization  of  the  network. 

•  For  100,000  entities,  MCGs  will  be  essential.  A  single  site  seems  unlikely  to 
need  to  receive  more  than  10,000  entities  even  in  future,  which  should  help 
bound  the  required  size  of  the  tail  circuits  and  capabilities  of  the  site  network 
equipment  and  computers. 

Comments  on  Specific  Presentations 

LADS 

The  LADS  presentation  was  highly  impressive.  Its  work  on  a  network  simulation 
using  real  data  and  SAF  (semi-automatic  forces)  is  most  valuable  and  needs  to  continue. 
The  impact  of  the  proposed  algorithms  on  simulation  fidelity  must  be  addressed.  Also,  the 
effectiveness  of  the  algorithms  should  be  confirmed  using  real-life  field  tests.  The  network 
simulation  needs  to  be  tested  against  real  network  experiments.  There  appears  to  be  good 
cooperation  and  division  of  labor  between  LADS  and  BBN.  The  On-Demand  Forwarding 
algorithm  may  be  more  promising  than  geographical.  There  is  no  need  to  have  both  LADS 
and  AT&T  look  at  TimeOuts.  More  detail  should  be  incorporated  into  the  network  traffic 
simulation.  Accurate  modeling  of  MCG  joins  and  leaves  may  not  be  a  1 994  issue,  given  the 
static  nature  of  NES  MCGs.  The  effect  of  compression  should  be  quantified. 

AT&T 

An  Advanced  Interface  Unit  (AIU)  or  “intelligent  gateway”  is  needed  to  reduce 
traffic,  but  the  implementation  is  very  difficult  It  was  not  clear  what  AT&T  was  proposing 
to  put  in  the  AIU.  There  appeared  to  be  some  risk  in  having  AT&T  develop  the  AIU  due  to 
the  lack  of  previous  experience  in  this  particular  area.  Due  to  changes  in  the  networking 
requirements,  the  AT&T  work  is  no  longer  as  relevant  to  ASTO  programs  as  when  the  orig- 


inal  contract  was  awarded;  hence,  some  of  the  comments  may  seem  more  negative  than  oth¬ 
erwise  would  have  been  expected. 

The  AT&T  recommendation  to  use  delta  state  updates  and  situational  information 
was  similar  to  LADS  geographical  filtering,  except  AT&T  proposed  to  reduce  the  number 
of  bits  in  the  protocol.  Using  delta  PDUs  may  introduce  simulation  compromises.  This  pre¬ 
sentation  had  too  much  emphasis  on  speculation  and  too  litde  experimental  science.  ^ 

General  Comments 

•  There  needs  to  be  another  component  in  the  Scaleability  program  analyses  than 
just  bottom-up,  although  there  is  the  danger  in  getting  only  fluff  with  a  top- 

down  approach.  STOW  will  need  to  use  multiple  techniques  in  parallel  to  * 

achieve  the  required  network  traffic  reduction,  but  the  use  of  multiple  tech¬ 
niques  vastly  complicates  the  simulators  and  networks. 

•  Some  things  are  network-wide  issues  that  cannot  be  solved  locally. 

•  STOW  should  look  carefully  at  DSI  transition. 

•  STOW  has  no  need  for  network  reservation  since  the  projected  number  of  T- 1 
tail  circuits  cannot  overload  a  T-3  backbone. 

•  STOW  does  need  to  be  able  to  set  priorities  for  traffic  coming  on/off  tail  circuits.  i 

•  NES  is  a  big  concern.  STOW  needs  a  more  detailed  network  simulation  incor¬ 
porating  NESs  to  determine  real  risks  and  throughput. 

•  Surprisingly  and  disconcertingly,  many  questions  that  ought  to  have  clear 

answers  did  not  Too  often  there  is  a  need  for  different  people  from  different  * 

organizations  to  get  together  to  answer  questions.  The  Scaleability  program  and 
the  DSI  have  a  problem  with  integration — determining  what  topics  to  be  inves¬ 
tigated  and  what  needs  to  be  coordinated. 

•  The  AIU  is  a  complicated,  sophisticated  box.  Multiple  people  are  working  on  * 

building  it  Functionality  continues  to  be  added  in,  but  STOW  needs  to  start 

building  AIUs  very  soon.  Some  proposed  bandwidth  reductions  are  really  DIS 
protocol  changes  that  could  possibly  be  added  into  AIUs. 

•  Using  satellites  with  delay  compensation  (through  the  use  of  dead  reckoning 
with  some  adjustments  for  propagation  time  delay)  to  minimize  the  impact  of 
latency  should  be  considered  as  a  replacement  for  always  using  terrestrial  net¬ 
work  links. 
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Comments  on  STOW  *94  and  *97  Scaleability  Issues 


David  R.  Cheriton 
Stanford  University 


A  major  constraint  on  STOW  ’94  is  the  security  requirement,  given  the  apparent 
necessity  to  use  the  Motorola  NES  to  satisfy  this  requirement  The  panel  only  really  came 
to  grips  with  this  limitation  on  the  second  day  of  presentations,  and  this  constr  aint  domi¬ 
nated  some  issues  that  were  presented  and  discussed  earlier. 

The  NES  introduces  some  major  uncertainties  into  the  program.  There  wer^  p.  opos- 
als  to  gang  a  number  of  NESs  together  to  increase  data  rates,  but  there  has  not  been  any 
experience  with  this  approach  to  date.  The  proposed  approach  did  not  increase  the  number 
of  multicast  groups  beyond  the  hoped-for  16  per  NES  because  each  is  conhgured  identical¬ 
ly  to  provide  flexible  load-sharing.  An  alternative  approach,  in  which  each  NES  at  a  site 
provides  a  separate  set  of  16  multicast  groups,  reducec  this  flexibility  of  load  sharing  but 
increases  the  number  of  multicast  groups.  Little  is  known  about  the  behavior  of  traffic  in 
this  configuration  either.  If  ganging  NESs  at  a  site  to  support  secure  traffic  close  to  the  T1 
speeds  of  the  line  is  cost  justified  (see  below),  study  is  required  to  better  understand  the 
behavior  of  the  multicast-based  traffic  reduction  techniques  within  this  network  configura¬ 
tion.  That  is,  how  well  do  16*k^  multicast  groups  or  so  per  site  reduce  traffic  (assuming  we 
have  k  NES  nodes  per  site),  and  can  this  traffic  be  expected  to  be  distributed  fairly  uniform¬ 
ly  across  the  groups  so  there  are  not  bottleneck  NES  nodes. 

The  value  of  k  (NESs  per  site)  was  estimated  as  having  to  be  around  6  to  10  to 
achieve  secure  bandwidth  that  matched  the  speeds  of  the  T1  site  tail  circuits.  This  number 
might  be  lowered  somewhat,  based  on  further  experiments,  but  one  possible  merit  in  using 
a  larger  value  of  k  is  providing  more  multicast  groups,  namely  16* k. 


’  [where  k  is  any  number.] 
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At  $17,000  per  node,  this  cost  would  represent  a  significant,  possibly  prohibitive, 
and  likely  throw-away  (after  STOW  ’94)  cost.  Given  the  major  cost  of  this  model,  one 
might  consider  non-trivial  alternatives.  One  further  alternative  traffic  reduction  technique 
that  was  raised  but  not  explored  in  detail  was  to  replicate  the  SAP  execution  at  each  site  so 
that  this  traffic  is  largely  removed  from  the  tail-circuits.  It  is  estimated  that  90  percent  of 
the  traffic  will  be  SAP  generated.  Buying  extra  computer  server  hardware  to  run  redundant 
SAP  as  each  site  and  reducing  the  capital  expenditures  on  NESs  seem  preferable  based  on 
the  reusability  of  some  computer  servers  after  STOW  ’94.  Por  example,  these  could  be 
additional  SGI  workstations  or  similar.  However,  this  direction  would  require  some  further 
study  and  development  to  coordinate  SAP  between  different  sites,  and  may  in  fact  involve 
modifications  to  the  SAP  to  increase  tlie  determinism  of  their  execution.  Work  in  this  area 
may  have  long  term  pay-off  in  providing  another  opportunity  for  trading  off  between  com¬ 
putation  and  communication,  an  issue  that  can  be  expected  to  be  relevant  in  ‘97,  given  the 
expected  continued  dependence  on  satellite  communication.  However,  it  should  be  empha¬ 
sized  that  this  is  an  area  that  would  require  effort  to  investigate  and  develop  as  an  option; 
not  all  of  the  panel  even  convinced  that  this  was  clearly  feasible.  Perhaps  an  effort  in  this 
area  could  be  considered  in  conjunction  with  Commander  McBride’s  program. 

Considering  STOW  ’97  briefly,  there  seems  to  be  a  potentially  excessive  trust  in 
ATM  technology  adequately  solving  all  the  bandwidth  problems.  Multicast  is  not  currently 
well- supported  in  the  ATM  standard,  ATM  has  not  been  tarriffed  in  general,  and  other  com¬ 
peting  technologies  may  come  along  or  be  necessary  to  use,  an  obvious  example  being  sat¬ 
ellite.  As  one  aspect  of  accommodating  these  uncertainties  with  the  network  technology, 
there  should  be  greater  emphasis  on  NSA  developing  a  network-technology  independent 
encryption  device  (N-TTED).  I  plan  to  submit  a  sketch  proposal  for  such  a  device  as  a  sep¬ 
arate  document. 

In  this  same  vein,  I  think  it  would  be  worth  further  investigation  of  delay  compen¬ 
sation  techniques  so  that  satellite  links  could  be  confidently  incorporated  in  the  future.  They 
appear  necessary  for  both  ‘94  and  ‘97.  The  dead  reckoning  mechanism  provides  a  basis  for 
delay  compensation  so  one  may  be  able  to  build  this  as  an  extension  of  this  existing  mech¬ 
anism. 

Further  on  STOW  ’97,  there  is  relatively  little  understood  about  the  scaleability 
problems  for  STOW  ’94  and  the  actual  behavior  of  10,000  entities.  We  should  try  to  keep 
our  options  as  open  as  possible  for  ‘97,  given  that  a  success  in  ‘94  and  further  progress  with 
DIS  (such  as  dynamic  terrain)  may  significantly  drive  up  the  requirements  for  ‘97  beyond 


A-4 


that  expected  from  purely  increasing  the  number  of  entities.  The  key  point  is  not  that  there 
is  a  problem  but  there  is  definitely  risk,  and  exploring  lots  of  options  and  keeping  them  open 
seem  like  the  best  approach  to  minimizing  these  risks.  As  one  example,  relatively  little 
attention  has  been  paid  in  the  performance  studies  to  the  effects  of  wide-area  perception 
units,  such  as  stealth  vehicles  and  JSTARS.  These  units  may  create  significant  load 
demands  on  the  network  beyond  that  of  normal  entities,  and  their  importance  may  increase 
by  ‘97. 

Considering  the  presentations  overall,  I  was  disappointed  that  the  presenters  did  not 
explicitly  address  fidelity  maintenance  in  most  instances  of  discussing  various  optimiza¬ 
tions.  Fidelity  must  be  considered  in  each  and  every  optimization,  not  because  it  is  neces¬ 
sarily  a  problem,  but  we  cannot  afford  to  lose  fidelity  beyond  some  tolerance  level.  I  also 
felt  that  the  presenters  did  not  in  general  distill  out  the  key  issues  and  solutions.  For  exam¬ 
ple,  we  were  dragged  through  grid-based  multicast  filtering  only  to  discover  that  so-called 
on-demand  forwarding  (ODF)  was  better,  and  then  only  to  discover  that  ODF  required  far 
more  multicast  groups  that  would  be  available  in  a  secure  network  in  ‘94.  The  latter  sug¬ 
gests  a  lack  of  communication  between  groups.  (I  pick  that  particular  example  because  van 
Hook,  as  one  of  the  strongest  presenters,  seems  more  diplomatic  to  pick  on.) 

Comments  on  Specific  Presentations 

van  Hook  (Loral) 

Overall,  a  fine  presentation.  However,  this  work  needs  to  consider  the  problem 
slightly  differently,  given  the  NES.  We  need  to  know  how  well  one  can  do  with  multicast- 
based  filtering  using  specific  numbers  of  groups  like  16,  32, 48,  64,  etc.,  corresponding  to 
that  expected  to  be  supported  by  the  NES.  The  study  presented  seemed  aimed  instead  at 
picking  the  best  approach,  viewing  multicast  group  as  basically  free. 

Miller  (AT&T) 

It  was  not  clear  what  contribution  AT&T  was  going  to  make,  and  1  was  left  with  the 
impression  that  this  group  was  not  going  to  be  able  to  deliver  anything  that  was  a  real  con¬ 
tribution  to  STOW  ’94  or  ‘97. 

Seeder  (BBN) 

1  failed  to  find  much  of  merit  in  this  presentation  for  several  reasons.  Firstly,  there 
seemed  to  be  a  rather  poorly  defined  focus  to  this  work.  1  could  not  really  understand  what 
they  were  tasked  to  do.  Secondly,  the  recommendations  on  congestion  control  seemed  like 
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rather  weak  opinions  with  no  scientific  or  engineering  backing.  I  even  reviewed  the  provid¬ 
ed  Castineya  and  Escabar  paper,  and  found  no  more  content.  For  example,  they  glibly  dis¬ 
card  FIFO  queuing  as  a  viable  approach  while,  at  the  present,  Internet- working  luminaries 
like  Dave  Clark  of  MIT  are  discovering  in  their  research  that  FIFO  queueing  has  some  very 
nice  properties  for  minimizing  average  delay  in  real-time  communications  settings.  It  is  not 
clear  who  is  right  or  wrong,  but  I  didn’t  get  a  lot  of  confidence  in  the  approach.  They  need 
data  before  throwing  about  raw  opinions,  especially  in  an  area  as  difficult  as  congestion 
control. 

The  measurements  of  multicast  join/leave  behavior  seemed  to  have  a  number  of 
strange  unexplained  characteristics,  such  as  some  very  short  join  times,  and  the  traces  being 
used  to  gather  the  data  seem  very  weak.  It  would  seem  better  to  integrate  this  work  with  the 
performance  evaluations  being  done  by  van  Hook  so  we  get  the  complete  picture,  rather 
than  two  independent  efforts  that  both  look  at  only  part  of  the  overall  load.  And  van  Hook 
seemed  better  positioned  to  carry  this  work  forward. 

Finally,  the  BBN  work  looking  at  protocol  options  for  DSlnet  seemed  extremely 
biased  in  favor  of  ST  II  which  BBN  developed.  It  is  clear  at  the  present  time  that  IP  multi¬ 
cast,  the  clear  competitor,  is  enjoying  far  greater  success  in  the  commercial  marketplace 
while  it  was  listed  as  “experimental”  versus  ST  II  being  listed  as  “available.”  I  also  felt  that 
the  amount  of  further  development  required  for  ST  D  in^lementations  to  actuaUy  support 
STOW  ’94  was  not  being  presented  accurately,  with  considerable  confusion  between  what 
the  protocol  was  supposed  to  do,  and  what  implementations  actually  did.  Personally,  1  feel 
that  IP  multicast  would  be  a  far  better  COTS-oriented  solution  for  DSlnet,  and  that  it  would 
be  feasible  to  acquire  COTS  IP  multicast  routrars  for  STOW  ’94  which  would  provide  a 
longer-term  investment  than  an  ST  II  solution.  However,  given  my  involvement  with  IP 
multicast,  this  opinion  is  likely  biased  as  well.  It  is  good  to  hear  that  Houston  Associates  is 
going  to  consider  this  issue  as  an  independent  agent,  but  it  does  appear  unlikely  that  they 
will  be  able  to  evaluate  the  options  adequately  within  the  short  time  frame  of  STOW  ’94 
deployment.  The  only  real  benefit  that  ST  n  appears  to  provide  over  IP  is  some  claimed 
ability  to  insulate  ST  II  traffic  from  IP  traffic.  I  think  that  this  level  of  resource  management 
can  be  accomplished  by  regulating  regular  IP  (multicast)  routers.  It  seems  unfortunate  to 
invest  in  non-standard  approaches  like  ST  II  for  one  exercise  when  a  COTS  solution  is 
available.  We  note  further  that  IP  multicast  is  being  actively  developed  and  refined  by  the 
Internet  community,  including  router  vendors,  to  satisfy  the  demands  of  real-time  video  and 
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audio.  These  demands  largely  subsume  those  for  DIS,  so  DIS  can  automatically  benefit 
from  the  commercial  directions  here  if  only  it  selects  IP  multicast. 

I’d  like  to  see  a  clearer  definition  of  BBN’s  role  in  the  networking  component  and 
more  careful  work  in  that  role  than  what  was  evidenced  by  the  presentation. 

Lou  Berger  (BBN) 

I  got  the  impression  that  he  was  just  providing  an  overview  of  DSInet,  and  could 
not  very  well  identify  or  articulate  the  key  issues  on  which  we  should  focus.  He  did  under 
tight  questioning  provide  some  good  insight  into  the  DSInet  and  its  problems  and  limita¬ 
tions.  I  wish  he  could  have  developed  his  own  candidate  list,  saving  us  some  time  in  teasing 
it  out  of  him. 

Overall,  it  would  seem  beneficial  to  have  someone  capable  taking  an  overall  view 
of  the  project  and  studies,  rather  like  this  review  panel  as  tried  to  do,  but  on  an  on-going 
basis.  Only  a  few  aspects  of  the  current  studies  seem  useful  and  there  are  many  issues  to 
consider  and  study,  as  pointed  out  above  and  by  the  other  review  members. 

Comments  on  David  Mills  review 

These  are  a  few  brief  reactions  to  David  Mills  review  input,  which  I  thought  was 
very  good  on  the  whole,  but  did  have  a  few  comments  that  either  in  my  interpretation  or  his 
intent,  I  have  some  question  on. 

1 .  I  feel  the  description  of  the  “current  technology’’  for  network  resource  manage¬ 
ment  is  far  too  optimistic.  Feirari’s  work  has  never  really  been  used  and  1  don’t 
think  is  really  practical  or  useful  for  the  problems  of  DIS.  Clark’s  work  is  far 
more  promising,  but  is  still  in  the  proposal  and  evaluation  stage.  I  think  it  is  very 
premature  to  “insist”  that  the  commercial  vendors  incorporate  this  stuff  into 
their  products.  They  have  ample  motivation  to  address  these  problems  for  other 
commercial  reasons.  And  I  believe  they  are  viewing  that  current  state  as  prema¬ 
ture  to  use  as  well.  I  would  also  mention  diat  tho-e  is  a  long  history  of  “resource 
management”  mechanisms  making  situations  worse  rather  than  better,  so  just 
because  we  have  the  need,  don’t  assume  that  current  medicine  will  make  you 
better. 

2.  1  didn’t  fiilly  undnstand  the  point  on  group  membership  and  the  recommenda¬ 
tion.  IP  multicast  provides  joins  within  roundtrip  times,  which  is  the  best  one 
can  do.  The  leave  latency  is  controlled  by  a  timeout,  which  defaults  to  90  sec- 
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onds.  One  could  clearly  tighten  that  up  substantially.  1  agree  that  there  may  be 
issues  to  explore  here,  but  let’s  make  sure  we  don’t  fund  a  whole  research  pro¬ 
gram  when  90  percent  of  the  benefit  would  be  gained  by  just  tightening  this  tim¬ 
eout  on  leaving  a  group.  Perhaps  more  strongl}’,  the  latter  parameter  adjustment 
should  be  tried  before  we  really  decide  we  have  a  problem.  It  is  not  clear  to  me 
that  there  is  one.  I  expect  one  does  have  to  tolerate  some  extra  level  of  network 
traffic  because  of  joining  and  leaving  not  being  instantaneous.  However,  I  don’t 
see  this  leading  to  spikes.  We  have  to  remember  that  the  multicast  techniques 
are  just  an  optimization  over  the  broadcast  approach  currently  being  used.  That 
is,  we  are  improving  on  what  is  there,  not  departing  from  some  absolute  opti¬ 
mum. 

3.  I  didn’t  understand  how  to  do  header  compression  without  changes  to  otherwise 
COTS  routers.  I  was  assuming,  and  understood,  that  the  backbone  net  capacity 
was  going  to  be  improved.  I  am  not  convinced  that  header  compression  is  going 
to  be  worth  the  pain  in  this  light,  although  I  really  do  not  understand  what  Mills 
has  in  mind  to  use  here.  Perhaps  this  should  be  examined  and  clarified  before 
the  recommendation  is  taken  too  seriously. 


A-8 


Scaleability  for  STOW  ‘94  &  ‘97 


Dale  B.  Henderson 
Los  Alamos  National  Laboratory 


The  scaleability  project  was  very  well  articulated  by  Commander  Dennis  McBride, 
whom  I  had  not  the  pleasure  of  meeting  before.  He  seemed  to  appreciate  both  the  opportu¬ 
nities  and  risks  to  his  tasks  of  10,000  and  100,000  entity  exercises.  The  solutions  which  he 
proposed  seem  to  be  the  likely  ones  to  consider.  That  of  which  I  am  a  little  unsure  is  the 
overall  context  in  which  these  solutions  are  to  apply.  While  perhaps  outside  of  or  at  the  edge 
of  the  charge  to  the  present  panel,  some  of  these  solutions  may  be  taken  to  compromise  the 
basic  genius  of  SIMNET.  Does  this  matter,  or  is  anything  really  being  sacrificed?  I  do  not 
know,  but  I  feel  that  the  questions  should  be  addressed.  Perhaps  Colonel  Reddy  would  have 
addressed  them  had  he  been  able  to  attend.  Perhaps  you  need  another  panel  or  a  funded 
(contractor?  FFRDC?)  study. 

To  me  the  basic  genius  of  SIMNET  is  the  idea  of  a  broadcast  state  message  com¬ 
bined  with  the  notion  of  dead  reckoning  to  reduce  the  network  traific.  The  various  new 
ideas  which  we  were  asked  to  consider  are  compromises  on  this  theme.  G  say  this  while 
acknowledging  having  been  a  member  of  the  previous  panel  which  worked  at  inventing 
some  of  the  ideas  under  discussion.)  A  simulation  employing  a  network  with  interest  filter¬ 
ing  or  fidelity  channels  or  interest  registration  is  a  lot  more  complicated  and  also  less  ele¬ 
gant  than  the  original  SIMNET.  The  panel  had  only  slight  briefings  on  and  no  internal 
discussion  of  the  computational  requirements  of  implementing  these  algorithms.  I  worry 
[about]  the  added  loads,  both  at  the  intelligent  gateways  and  at  the  entities  may  be  under¬ 
estimated.  This  should,  however,  be  explored  with  the  tool  set  discussed  by  Mr.  van  Hook. 

To  address  whether  or  not  the  network  traffic  reducing  methods  compromise  the 
mission  of  the  exercise  or  simulation  requires  that  we  consider  what  this  mission  is.  Clearly 
it  is  not  just  to  put  10,000  or  100,000  entities  on  a  network;  that  is  too  much  like  climbing 
a  mountain  because  it  exists.  To  justify  any  of  the  filtering  schemes  requires  consideration 


of  the  negative  e  'fects,  if  any,  on  the  credibility  and  applicability  of  the  simulation.  With 
SIMNET  the  goals  were  training.  Training  value  should  be  measurable  through  after-action 
reports,  testing,  and  other  measures.  Now  that  we  are  talking  seriously  about  other  applica¬ 
tions,  we  need  to  include  a  broader  assessment  of  the  very  efficacy  of  the  simulation,  espe¬ 
cially  as  we  introduce  some  of  these  complicated  mechanisms  to  enable  scaling.  By  the 
way,  some  of  the  problems  of  using  “time  warp”  schemes  in  parallel  computer-based  sim¬ 
ulation  are  analogous. 

The  role  and  dominance  of  semi-automatic  forces  [SAP]  raises  similar  philosophi¬ 
cal  questions.  As  I  thought  I  understood  the  role  of  semiautomatic  forces  in  SIMNET,  they 
served  to  add  data  richness  as  an  enhanced  background  to  a  fundamentally  manned  system 
of  trainers.  Now,  with  the  larger  numbers  of  total  entities,  they  (SAP)  are  coming  to  pre¬ 
dominate.  This  brings  us  to  that  which  I  know  more  about:  very-many-instance  simulations 
with  much  (or  all)  of  the  communication  within  the  shared  memory  of  a  central  supercom¬ 
puter.  Add  a  few  manned  instances  of  objects  to  one  of  these  through  external  interfaces, 

I 

and  one  comes  to  the  same  point  from  the  other  direction.  But  looking  at  it  this  way  brings 
along  questions  (typically  addressed  in  the  non-DIS/DSI  world)  of  significance,  fidelity, 
applicability,  and  accreditation  which  deserve  answers. 

Such  consideration  should  include  the  whole  system.  An  intelligent  gateway,  for  i 

instance,  is  not  just  a  passive  service  but  is  an  integral  part  of  the  simulation  engine.  Interest 
registration  and  other  ideas  for  tailoring  the  traffic  move  important  decisions  away  from  the 
entities  which  are  best  prepared  to  consider  their  effects. 

Although  we  did  not  discuss  them,  there  were  suggestions  of  reductions  in  the  arith-  < 

metic  precision  of  field  quantities,  from  64  to  only  16  bits.  1  occasionally  worry  that  any 
simulation  is  an  iterated  rem£q)ping  of  data  into  its  own  space.  Ito'ated  remaps  can  be 
numerically  unstable;  this  is  the  basis  of  numerical  chaos.  The  conditions  for  the  numerical 
instability  of  the  Lanchester  equations  are  known  and  demonstrated.  Por  more  complex  < 

simulations,  we  (I)  can’t  guess,  but  I  would  worry  that  reducing  numerical  significance  can 
not  help. 

I  support  the  group-think  conclusion  of  our  final  session  that  the  1994  goal  of 
10,(XX)  entities  operating  in  a  secure  environment  is  not  possible.  The  arithmetic  is  simple  * 

and  compelling.  Thinking  about  the  ARPA  presentations,  I  su^a  that  they  already  knew 
this  and  were  really  looking  for  our  confirmation. 


< 
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While  the  government  seemed  to  have  its  act  together,  the  contractors  left  me  with 
more  of  a  mixed  impression.  I  share  the  group’s  impression  that  AT&T  seemed  especially 
weak.  Could  it  be  that  they  are  really  just  getting  started? 

I  would  rate  van  Hook’s  contractor  briefing  as  best  in  that  the  tool  set  appears  to  be 
the  right  capability  at  the  right  time.  The  results  should  replace  our  collective  guesses  about 
traffic  reduction  with  real  information.  That  they  can  replay  collected  data  from  past  exer¬ 
cises  will  make  the  results  more  convincing.  The  addition  of  some  kind  [of)  a  loop  closure 
might  also  help:  how  would  any  of  the  network  performance  enhancing  algorithms  affect 
the  simulation?  Generally  the  network  measures  of  performance  need  to  be  linked  to  the 
scenario  bottom  line. 

Perhaps  this  tool  could  be  extended  to  address  some  of  the  questions  of  validity  and 
accreditation  for  simulations  mnning  under  the  various  filtering  and  other  network 
enhancement  schemes.  Fidelity  channels,  on-demand  forwarding,  and  interest  registration 
each  raise  questions  which  might  be  addressed.  Even  multicast  grids  should  be  proven  safe 
before  they  are  utilized  in  a  big,  expensive  exercise. 

The  DSI  understanding  discussed  by  Lou  Berger  appears  to  be  essential.  He  seems 
to  have  a  more  comprehensive  knowledge  than  anyone  else.  Yet,  the  panel  easily  asked  rel¬ 
evant  questions  to  which  the  answers  were  not  immediate  or  clear;  the  obvious  (hopefully 
incorrect)  conclusion  is  that  nobody  quite  knows  the  answers.  An  example  of  this  was  our 
questions  about  the  addressing  through  the  T-20s  and  NESs — did  we  finally  get  it  straight 
or  did  we  just  quit  asking  about  it?  I  am  unsure.  The  tool  set  should  also  help  with  these 
issues.  Does  it  include  the  Network  Encryption  System?  If  not,  the  extension  could  be  very 
important 

In  this  regard,  why  did  we  get  such  an  impression  that  the  DSI  is  more  of  a  hostage 
to  the  NSA  than  in  paitnership  with  the  NSA?  I  once  raised  an  issue  of  key  policy  to  the 
NSA  and  received  a  very  positive  and  helpful  response.  (Furthermore,  in  comparison  with 
STOW,  the  government  had  much  less  on  the  table  in  my  case.)  Having  NSA  participation 
at  the  review  was  probably  a  good  step  by  Lynn  Flynn  in  establishing  a  feeling  of  being  in 
it  together.  Maybe  you  should  have  invited  someone  not  directly  involved  from  NSA,  per¬ 
haps  its  Chief  Scientist  or  the  direaor  of  the  SRC,  to  be  part  of  our  panel.  There  is  always 
next  time. 

I  want  to  repeat  a  specific  remark  from  the  discussion.  I  suggest  that  the  Magic  Car¬ 
pet  or  Plan  View  devices  ought  to  be  reprogrammed  so  that  the  data  are  filtered  locally  (gen- 


erally  West  Coast?)  according  to  commands  from  the  viewer  (generally  East  Coast?)  and 
only  images  be  transmitted  back.  (Even  if  these  stealth  viewers  are  directly  on  the  T-3,  the 
data  must  be  filtered  somewhere.)  As  best  I  now  understand,  these  viewing  platforms  are 
the  most  data-demanding  entities,  and  they  clearly  do  not  feedback  into  the  scenario’s  bot¬ 
tom  line,  so  that  none  of  my  questions  of  significance  would  apply. 
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Issues  of  Scaleability  for  Stow 


Duncan  C.  Miller 
MIT  Lincoln  Laboratory 


Overall  Conclusions 

The  briefings  and  panel  discussions  focused  primarily  on  the  goal  of  a  10,000  entity 
exercise  for  STOW  ’94.  Secondarily,  we  considered  the  applicability  of  these  approaches 
for  STOW  ’97. 

The  consensus  of  the  panel  was  that  a  10,000  entity  exercise  a  year  from  now  is 
extremely  risky  if  it  must  be  conducted  on  a  secure  network.  The  principal  constraint  is 
imposed  by  the  NES  devices. 

In  a  worst-case,  “broadcast”  approach,  in  which  all  exercise  traffic  is  sent  to  every 
site,  the  T- 1  tail  circuits  to  the  sites  would  be  overloaded  by  a  factor  of  about  10.  Therefore, 
it  is  clear  that  an  efficient  filtering  mechanism,  probably  based  on  multicast  groups,  wiU  be 
necessary.  The  algorithms  used  must  be  clever  enough  to  achieve  a  90%  traffic  reduction 
on  each  tail  circuit  during  the  peak  load  periods  of  the  exercise. 

The  principal  point  of  risk  is  that  the  Motorola  NES  devices,  which  are  currently 
the  only  approved  end-to-end  encryption  devices  for  such  applications,  will  not  be  able  to 
suppon  the  necessary  numbers  of  multicast  groups.  If  clever  workarounds  cannot  be  found 
to  the  constraints  imposed  by  the  NES  devices,  and  the  requirement  for  a  secure  network 
cannot  be  relaxed,  there  seem  to  be  only  two  fallback  paths  to  pursue: 

1 .  Reduce  the  magnitude  of  the  exercise  by  a  factor  of  three  or  four. 

2.  Procure  multiple  NES  devices  (and  perhaps  multiple  T- 1  lines)  for  each  site.  We 
suspect  that  as  many  as  10  encryption  devices  may  need  to  be  ganged  together 
to  support  the  needed  number  of  multicast  groups.  This  approach  might  require 
130  NES  devices  for  the  13  planned  sites. 


In  my  opinion,  if  effective  multicast  grouping  algorithms  can  be  developed,  the  per- 
site  traffic  loads  for  a  100,000  entity  exercise  in  1997  will  not  be  substantially  greater  than 
for  a  10,000  entity  exercise.  My  basis  for  this  opinion  is  that  on  a  real  battlefield,  most  units 
reach  a  saturation  point  in  the  number  of  other  units  they  can  consider.  A  company-size  unit 
will  continue  to  focus  on  approximately  the  same  number  of  nearby  company-,  battalion-, 
and  regimental-size  units  no  matter  how  large  the  total  battlespace  becomes. 

Although  STOW  ’97  represented  only  a  secondary  focus  of  our  discussions,  we  felt 
reasonably  comfortable  that  ATM  technology,  wideband  fiber-optic  transmission  media, 
and  high-speed  interfaces  will  be  sufficiently  well  developed  within  the  next  three  years 
that  exercises  involving  an  aggregate  traffic  load  of  100,000  packets  per  second  will  be  fea¬ 
sible.  Whatever  lessons  are  learned  during  STOW  ’94  can  be  readily  applied  to  the  imple¬ 
mentation  of  STOW  ’97. 

Comments  on  Specific  Presentations 

Dan  van  Hook  (Loral) 

Dan  van  Hook’s  presentation  on  simulation  tools  and  algorithm  evaluation  was  fas¬ 
cinating.  It  has  been  clear  for  a  long  time  that  filtering  algorithms  will  be  critical  in  deter¬ 
mining  how  far  distributed  simulation  can  be  scaled  up,  and  many  of  us  have  speculated 
vigorously  about  what  degree  of  traffic  reduction  can  be  achieved  in  realistic  exercise  sce¬ 
narios.  Dan’s  is  the  first  data  that  has  come  out  that  can  be  used  to  begin  to  calibrate  our 
expectations.  It  is  exciting  to  see  that  traffic  reductions  by  factors  of  three  to  five  or  more 
seem  possible.  The  “On-Demand  Filtering”  plus  “Fidelity  Channel”  plus  “Dead  Entity 
Server”  combination  of  algorithms  seems  particularly  promising. 

It  is  essential  that  these  tools  be  brought  as  quickly  as  possible  to  a  state  where  var¬ 
ious  organizations  can  begin  to  use  them  and  to  explore  hypothetical  scenarios  and  algo¬ 
rithms.  Then  an  objective  dialog  can  begin  regarding  the  tradeoffs  involved  in  traffic 
control  algorithms,  network  topology,  and  exercise  scenario  design.  Probably  the  most  cru¬ 
cial  issue  that  needs  to  be  understood  is  that  of  the  dynamics  of  multicast  subscription  man¬ 
agement  under  alternative  assumptions.  It  may  weU  be  that  the  best  approach  in  this  area 
has  not  yet  been  conceived,  so  it  is  important  to  get  additional  creative  people  thinking 
about  it 
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Bill  Miller  (AT&T) 


This  presentation  was  less  impressive.  We  heard  very  little  by  way  of  innovative 
approaches  for  saving  communications  bandwidth.  It  was  suggested  that  changes  in  the 
DIS  protocols  could  produce  more  compact  PDUs  by  transmitting  incremental  changes  in 
vehicle  state  rather  than  the  vehicle  state  and  its  derivatives.  This  is  undoubtedly  true,  but 
it  would  require  a  fundamental  modification  to  die  approach  now  codified  in  IEEE  1278- 
1993.  If  this  idea  were  implemented,  it  would  be  better  to  intervene  at  a  local  area  network 
gateway  to  retransmit  the  more  compact  PDUs  over  the  >^de  Area  Net  and  reconstitute 
them  in  standard  form  at  the  receiving  LANs. 

It  was  also  suggested  that  derivatives  could  be  inferred  at  a  receiving  node  rather 
than  transmitted  explicitly.  This  proposal,  while  technically  possible,  reveals  a  serious  mis¬ 
understanding  of  the  dead  reckoning  concept 

When  a  pilot  opens  his  throttle  and  initiates  an  acceleration  of  his  aircraft  his  sim¬ 
ulator  knows  immediately  that  the  vehicle’s  time  derivatives  have  changed.  By  transmitting 
the  new  state  vector  derivatives  to  other  entities  immediately,  they  can  initiate  an  extrapo¬ 
lation  of  the  future  state  of  his  aircraft  that  will  remain  within  discrepancy  threshold  toler¬ 
ances  for  a  significantly  longer  time  than  a  simple,  constant  velocity  extrapolation.  Under 
the  proposed  alternative  scheme,  after  a  sufficient  period  of  time,  the  receiving  nodes  could 
infer  and  begin  to  make  use  of  the  aircraft’s  acceleration,  but  not  until  many  unnecessary 
PDUs  had  been  transmitted.  This  scheme  is  infmor  to  a  higher-order  dead  reckoning 
extrapolation  ^roach  in  terms  of  information  transmission  under  almost  all  circumstanc¬ 
es. 

The  other  AT&T  responsibility  described  to  us  was  the  development  of  detailed 
specifications  for  the  communications  interface  gear.  We  heard  nothing  to  suggest  that  this 
was  more  than  a  straightforward  systems  engineering  effort  that  could  be  successfully 
undertaken  by  any  number  of  SETA  organizations.  It  was  not  clear  why  the  capabilities  of 
AT&T  Bell  Labs  were  required. 

Josh  Seeger  and  Lou  Berger  (BBN) 

Josh  Seeger  presented  a  summary  of  the  network  modeling  being  carried  out  in  con¬ 
junction  with  the  LADS  application-level  modeling.  This  work  appeared  to  be  well  inte¬ 
grated  and  quite  useful  for  predicting  performance  bottlenecks  for  STOW  ’94.  The  most 
critical  technical  issue  in  this  area  is  the  dynamic  multicast  group  subscription/desubscrip¬ 
tion  rates  that  result  from  various  grouping  algorithms.  The  preliminary  data  presented  was 


intriguing  but  rough.  There  seemed  to  be  some  anomalies,  especially  in  the  numbers  of 
entities  that  maintained  a  subscription  to  a  particular  multicast  group  for  less  than  hve  sec¬ 
onds.  My  intuition  tells  me  that  the  numbers  we  were  shown  were  much  higher  than  could 
be  accounted  for  by  a  few  fast-moving  aircraft  and  by  entities  grazing  each  other’s  areas  of 
interest.  The  algorithms  being  simulated  should  be  checked  for  dynamic  stability. 

Lou  Berger’s  presentation  on  the  state  of  the  Defense  Simulation  Internet  turned  out 
to  be  the  focus  of  most  of  the  “bad  news’’  reflected  in  the  consensus  summarized  above. 

Here  was  where  we  discussed  security  issues  and  the  serious  limitations  of  the  NES  devic¬ 
es.  Here  was  also  where  the  “religious  issues’’  of  ST  vs.  IP  multicasting  were  debated.  My 
overall  perspective  on  these  issues  is  that  we  must  remember  that  we  are  dealing  with  some 
near-term  problems  for  STOW  ’94  that  we  expect  to  be  resolved  by  hardware  and  software 
developments  over  the  next  two  or  three  years.  ARPA  should  therefore  resist  pouring  more 
resources  than  are  necessary  into  short-term  solutions  to  these  problems.  Other  factors  are 
at  play,  both  in  terms  of  market  forces  for  comimicial  communications  and  in  DoD  require¬ 
ments  for  secure  communications,  that  will  dominate  the  specific  requirements  of  distrib¬ 
uted  simulation  in  general  and  STOW  in  particular. 

The  best  strategy  seems  to  be  to  do  whatever  can  be  done  with  the  existing  hardware 
and  software  for  STOW  ’94  while  closely  monitoring  developments  in  other  technical  arc-  , 

nas.  If,  as  Dave  Cheriton  argues,  other  forces  are  working  to  make  IP  multicasting  an  indus¬ 
try  standard,  that’s  fine.  The  DIS  community  should  cooperate  in  this  development,  making 
sure  that  our  unique  needs  for  highly  dynamic  multicast  groups  are  understood  and 
addressed  by  the  larger  conununity.  (A  modest  level  of  support  for  ST-II  multicasting  may  , 

be  the  best  guarantee  that  the  larger  community  will  pay  attention  to  these  needs.)  Similarly, 
we  must  make  sure  that  NSA  understands  our  specific  needs  for  higher-bandwidth  end-to- 
end  encryption  devices  that  can  handle  large  numbers  of  small  packets  being  routed  to  mul¬ 
tiple  destinations.  ^ 

Finally,  I  must  add  that  the  prevailing  view  that  ATM  will  resolve  all  these  problems 
within  the  next  two  or  three  years  leaves  me  somewhat  apprehensive.  I  don’t  doubt  that  they 
will  be  resolved,  but  I  suspect  the  solutions  will  not  be  handed  to  us  on  a  platter.  We  will 
have  to  speak  up  forcefully  to  make  sure  our  needs  are  being  adequately  considered  in  a  < 

markeq)lace  that  is  being  driven  by  other  forces. 


( 
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Summary  of  STOW  Panel  Review 


David  L.  Mills 
University  of  Delaware 


Overall  Conclusions 

On  August  19  and  20,  1993,  a  Synthetic  Theater  of  War  (STOW)  program  review 
was  held  at  the  Institute  of  Defense  Analysis  in  Alexandria,  Virginia.  The  review  panel,  of 
which  I  was  a  member,  met  to  hear  technical  briefings  on  the  STOW  program,  specifically 
on  the  demonstrations  to  be  held  in  FY94  and  another  in  FY97.  Briefings  were  presented 
by  Loral  Advanced  Distributed  Simulation  (LADS),  American  Telephone  and  Telegraph 
(AT&T),  and  Bolt,  Beranek  Newman  (BBN).  The  following  are  my  impressions  on  these 
and  related  issues. 

The  BBN  briefing  emphasized  network-layer  modeling  while  the  LADS  briefing 
emphasized  appLcation-layer  modeling.  These  briefings  included  considerable  detail  and 
convincing  arguments.  The  LADS  briefing,  in  particular,  demonstrated  well-founded 
design  and  implementation  strategies  and  understanding  of  the  issues.  The  AT&T  briefing 
described  candidate  gateway  designs,  compression  algorithms,  and  dead-reckoning  algo¬ 
rithm  improvements,  but  was  generally  less  effective  than  the  others.  The  BBN  briefing 
raised  painful  issues  of  near-term  design  and  implementation  shortfalls  which  raise  real 
doubt  that  the  planned  level  of  10,000  entities  at  15  sites  is  sustainable  in  STOW  ’94. 

My  overall  assessment  on  the  technical  direction  of  the  STOW  program  is  mixed 
in  view  of  the  briefings  and  related  background  information.  On  one  hand,  it  is  apparent 
that  the  networking  technology  issues  mentioned  in  previous  review  panels  are  being 
addressed  and  studied  on  their  own  merit.  The  LADS  briefing  demonstrated  that  on- 
demand  forwarding,  fidelity  channels,  and  dead-entity  servers  have  the  potential  to  reduce 
the  network  traffic  by  factors  of  between  3  and  5,  at  least  in  certain  topologies  and  traffic 
scenarios.  However,  in  themselves  these  schemes  do  not  appear  capable  of  traffic  reduc¬ 
tions  by  factors  of  10  or  more,  as  may  be  required  by  STOW  configurati'^ns.  The  AT&T 
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briefing,  while  considering  some  issues  not  evident  in  the  LADS  briefing,  did  not  bring 
comfort  that  sufficient  coding  gain,  delta  PDUs,  or  dead-reckoning  improvements  would 
make  up  the  shortfall. 

I  don’t  think  it  reasonable  in  a  report  such  as  this  to  offer  specific  guidance  on  tech¬ 
nical  issues,  such  as  how  many  NES  are  required  and  what  available  options  should  be  pur¬ 
sued.  I  can,  however,  comment  on  the  design  philosophy  and  likely  outcome,  should  certain 
design  approaches  be  pursued.  In  anticipation  of  the  commentary  to  follow,  it  is  clear  that 
no  magic  bullet  is  likely  to  do  the  job  of  reducing  backbone  and  tail  traffic  by  a  factor  of  at 
least  10,  as  required  by  a  quick  analysis  of  the  projected  topology  of  STOW  ’94.  Rather,  it 
is  likely  that  this  goal  will  be  achieved,  if  it  is  achieved  at  all,  by  a  systematic  assault  on 
minor  inefficiencies,  each  of  which  may  contribute  a  small  fraction  of  the  overall  budget. 

Network  Model  and  Assumptions 

First,  I  assume  there  is  no  avoiding  encryption  for  end-end  data  transmitted  over  the 
WAN.  However,  if  commercial  WAN  providers  are  to  play  and  promiscuous  broadcast  is 
to  be  avoided,  there  has  to  be  some  way  for  the  (classified)  simulation  community  to  inform 
the  (unclassified)  network  providers  about  the  protocol  data  unit  (PDU)  address  and  service 
class,  in  order  to  route  the  traffic  along  multicast  trees  and  perform  whatever  traffic  mitiga¬ 
tion  is  required.  Following  the  SDNS  model  proposed  by  NSA,  which  I  would  assume  is 
politically  correct,  the  only  way  to  do  that  is  in  the  unprotected  PDU  header  which  can  be 
IP,  ISO,  or  even  ST-n.  Furthermore,  I  assume  the  SDNS  PDU  is  manufactured  at  the  point 
of  entry  from  the  LAN  to  the  WAN  and  that  the  PDU  is  protected  by  a  message  digest,  [and] 
so  cannot  be  modified  en  route.  Therefore,  the  only  thing  the  WAN  can  do  is  forward,  rep¬ 
licate,  or  discard  the  PDU.  Of  course,  PDUs  can  be  aggregated  by  encapsulation  as  long  as 
the  origir  al  format  is  restored  upon  exit  Note  that  there  may  be  some  argument  over  the 
degree  to  which  the  application  security  is  compromised  by  the  unprotected  PDU  header, 
which  admits  of  traffic  analysis. 

I  assuiiK  most  of  the  lessons  learned  in  current  practice  and  experiment  have  been 
internalized  in  the  network  design.  The  tra^c  to  be  handled  at  the  entrance  and  exit  of  the 
WAN  is  estimated  at  about  800  2,000'bit  packets  per  second,  which  is  not  high  by  current 
standards.  However,  the  encryption  devices  may  not  support  this  rate,  which  is  surprising 
when  compared  with  similar  commercial  devices  which  operate  at  much  higher  speeds.  I 
believe  that  admission  controls  will  be  necessary  to  prevent  network  collapse  by  denying 
service  or  group  formation,  that  leaky-bucket  traffic  grooming  will  be  necessary  to  reduce 
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the  occurrence  of  timeouts  due  to  PDU  bunching  and  timeouts,  while  multiple-class,  round- 
robin  scheduling  will  be  necessary  to  mitigate  delay  bounds  on  the  basis  of  absolute  or  sta¬ 
tistical  guarantees  or  no  guarantee  at  all. 

We  agreed  at  the  meeting  that  these  features  of  this  type  will  be  necessary  in  a  fully 
developed  DSI,  whether  or  not  the  network  is  dedicated  to  the  simulation  mission.  Current 
technology  based  on  the  work  of  Ferrari,  Clark,  and  others  suggests  a  design  based  on  these 
principles  is  practical  and  quite  likely  would  be  an  important  factor  in  ensuring  success  of 
the  STOW  ’94  simulation.  Of  particular  current  interest  is  the  guaranteed/stochastic/best- 
effort  delay-bound  model  which  maps  nicely  to  the  t5q)es  of  data  used  in  STOW  simula¬ 
tions. 

I  believe  it  necessary  that  class-oriented  service  policies  be  implemented  in  the  net¬ 
work.  These  policies  provide  for  a  set  of  priority  classes,  with  each  class  serviced  by  dead¬ 
line  and/or  round-robin.  This  may  be  the  single  most  important  feature  allowing  guaranteed 
deadlines  for  such  things  as  impact  reports,  stochastic  deadlines  for  position  reports,  and 
best  effort  for  other  traffic.  These  policies  also  provide  the  framework  for  an  engineered 
scheme  to  discard  excess  traffic  upon  overload. 

Recommendation:  Make  sure  the  vendors  are  aware  of  admission  control,  leaky- 
bucket  and  class-oriented  service  principles,  and  insist  they  be  incorporated  in  the  routers, 
encryption  devices,  and  end  systems. 

Simulator  Simulation 

In  my  opinion  the  most  important  development  to  come  from  the  briefings  is  a  sim¬ 
ulator  for  the  simulation  system  itself.  This  is  a  program  which  emulates  the  network  oper¬ 
ations  induced  by  a  functioning  discrete-time  simulation.  It  embodies  the  PDU 
compression,  multicasting,  and  forwarding  algorithms  of  the  WAN,  and  can  be  implement¬ 
ed  either  as  a  synthetic  simulator  driven  by  software  scripts  or  as  an  intelligent  local  router 
joining  Ethernet  segments,  for  example. 

With  this  tool  it  is  possible  to  determine  the  efrectiveness  of  various  compression 
schemes,  multicast  protocols,  and  class-oriented  scheduling.  If  my  conclusion  [is]  that 
there  is  no  magic  bullet  and  that  substantial  reductions  in  traffic  level  are  possible  only  by 
a  series  of  incremental  improvements,  this  tool  will  be  critical  to  the  mission  success. 

Recommendation:  Give  the  simulator-simulator  area  [an]  increased  emphasis  in 
tasking  statements  and  support  the  construction  of  a  suitable  intelligent  local  router  based 
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on  an  ordinary  workstation  and  supporting  toolkit.  A  possible  point  of  departure  could  be 
the  Sun  workstation  routers  developed  for  the  DARTnet  research  network  for  ARPA. 

SAFOR  Servers 

One  of  the  things  that  caught  my  interest  during  the  meeting  is  the  observation  that 
SAFOR  traffic  accounts  for  90%  of  the  total.  If  so,  the  effort  to  reduce  traffic  levels  would 
be  most  effective  if  concentrated  in  this  area.  Presumably,  SAFOR  traffic  is  generated  by  a 
suite  of  replicated  algorithms  in  each  entity  and  orchestrated  by  an  operator.  This  suggests 
an  approach  in  which  a  state  machine  representing  each  operator  is  instantiated  in  all  enti¬ 
ties.  Its  operations  could  be  managed  by  a  low-rate  protocol  that  avoids  the  transmission  of 
entity-state  PDUs  on  the  WAN.  This  is  the  same  kind  of  idea  that  leads  to  the  dead-entity 
server. 

There  may  be  applications  of  the  replicated  state-machine  approach  other  than  dead 
entities  and  SAFOR,  such  as  weather  and  terrain  changes,  craters,  etc.  1  believe  that  the 
benefits  of  such  techniques  as  on-demand  forwarding,  fidelity  channels,  and  compression 
may  well  have  approached  their  limits  and  the  replicated  state-machine  approach  may  be 
the  single  most  useful  tool  remaining. 

Recommendation:  Pursue  the  replicated  state  machine  approach  with  SAFOR, 
hunt  for  other  applications  that  yield  to  this  approach,  [and]  generate  a  suitable  model,  pro¬ 
tocol,  and  PDU  encoding  to  generalize  the  application. 

On<Demand  Forwarding 

I  have  some  concern  that  on-demand  forwarding,  which  can  be  described  as  receiv¬ 
er-directed  multicasting,  will  result  in  a  significant  reduction  in  traffic  for  the  STOW  ’94 
simulation.  My  reasoning  for  this  is  as  follows.  The  total  PDU  flux  emitted  by  all  entities 
in  the  STOW  ’94  simulation  is  expected  in  the  order  of  30  Mbps  with  10,000  entities  at  15 
sites.  For  the  purposes  of  discussion,  assume  the  emissions  of  all  sites  are  substantially 
equal  and  that  coding  and  aggregation  gains  permit  the  use  of  a  1 .544  Mbps  T1  tail  circuit. 
It  follows  that  this  pipe  will  probably  be  fully  loaded. 

In  the  topology  anticipated  for  the  STOW  ’94  simulation,  the  sites  will  not  be  richly 
connected  and  will  probably  take  the  form  of  a  linear  network  or  ring.  On  the  assumption 
that  the  backbone  will  probably  be  assembled  from  1.544-Mbps  T1  circuits,  there  will  be 
traffic  bottlenecks  where  the  tails  join  the  backbone.  It  does  not  seem  likely  that  every  sin¬ 
gle  member  of  a  group  will  be  at  a  particular  site,  so  there  will  be  at  least  some  degree  of 


A-20 


PDU  replication  in  the  backbone.  This  will  certainly  limit  the  effectiveness  of  the  spanning- 
tree  pruning  feature  of  on-demand  forwarding.  Since  the  backbone  is  not  likely  to  be  richly 
connected,  I  conclude  a  T1  backbone  is  not  likely  to  support  the  full  STOW  ’94  mission  no 
matter  how  the  multicast  groups  are  formed  or  how  the  spanning  trees  are  pruned. 

Recommendation:  Increase  the  capacity  of  the  backbone  network  either  by 
increased  fabric  speed  or  richer  connectivity.  Make  sure  admission  controls  are  in  place  and 
that  class-oriented  discard  policies  are  in  effect 

Group  Membership 

Now,  assume  that  some  magic  multicasting  scheme  now  in  use  or  yet  to  be  discov¬ 
ered  is  available  and  does  the  job;  that  is,  a  particular  traffic  flow  can  be  supported  by  real¬ 
izable  bucket  parameters,  routing  tables,  replication  points,  and  class  policies.  When  an 
entity  moves  from  one  multicast  group  to  another,  a  group-management  protocol  must  react 
quickly.  To  avoid  dropouts  as  the  tables  are  being  adjusted,  the  entity  will  probably  belong 
to  both  groups  for  some  interval  spanning  the  reaction  time  of  the  protocol.  This  is  likely 
to  induce  a  traffic  spike  in  the  network  for  at  least  that  period,  as  well  as  complicate  the 
hand-over  protocol. 

Unless  the  hand-over  protocol  is  incorporated  directly  in  the  design  of  the  multicast 
routing  algorithm,  there  will  be  additional  delay  as  the  routing  tables  are  adjusted  through¬ 
out  the  network.  Current  commercial  networking  practice  is  to  use  a  standard  link-state 
routing  protocol  like  ISO  IS-IS  for  the  network  routing  fabric  and  an  encapsulation  overlay 
with  a  special-purpose  multicast  routing  protocol  to  determine  routing  and  replication 
points.  In  designs  known  to  me,  the  group-management  functions  are  handled  by  a  dedicat¬ 
ed  protocol  layered  on  the  routing  substrate.  The  required  speed  of  response  for  a  real-time 
system  would  seem  to  require  that  the  link-state  routing,  multicast  routing/replication,  and 
group-management  functions  must  be  implemented  as  a  functional  unit.  While  protocols 
such  as  ST-n  and  IP  multicast  address  one  or  more  of  these  issues,  they  do  not  address  them 
all  in  an  integrated  way. 

Work  is  needed  toward  a  comprehensive,  real-time,  multicast  group  management; 
admission  control;  and  routing  management  protocol.  While  it  is  unlikely  that  the  develop¬ 
ment  and  implementation  of  such  a  protocol  could  be  completed  by  the  time  of  STOW  ’94, 
it  will  be  necessary  in  the  long  run  and  should  be  pursued. 
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Data/Header  Compression 

There  has  been  some  attention  to  the  issue  of  data  compression,  either  through  the 
use  of  variable-length  encoding  or  motion  compensations  as  described  by  AT&T.  If  the 
average  PDU  size  is  2,000  bits,  which  seems  to  me  on  the  high  side,  it  would  make  sense 
to  be  somewhat  more  aggressive  in  this  area;  however,  the  issue  seems  not  to  have  stirred 
the  imagination  of  the  standards  community  at  best.  However,  my  commentary  here  is  pri¬ 
marily  aimed  at  the  WAN  and  its  routers  which  see  the  PDU  as  a  protected  (encrypted)  data 
area  and  an  unprotected  header. 

Since  the  encryption  operation  effectively  randomizes  the  data  area,  it  is  unlikely 
that  any  compression  algorithm  can  reduce  its  size;  however,  this  leaves  open  the  issue  of 
header  compression  which  uses  an  associative  table  to  map  the  PDU  header  upon  entry  to 
a  minimal  header  for  transmission  through  the  network,  then  restores  it  upon  exit.  The  asso¬ 
ciative  table  must  be  constructed  either  at  flow  setup  or  on  the  fly,  with  timeout  as  in  other 
protocols  such  as  ARP.  From  experience  in  the  Internet,  these  techniques  could  reduce  the 
backbone  and  tail  traffic  by  5  to  10%. 

Recommoidation:  This  is  a  relatively  easy  thing  to  do  and  can  be  incorporated  at 
the  network  or  physical  layer  of  the  network  software.  It  is  an  ideal  candidate  to  be  explored 
using  the  simulator  mentioned  above. 
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Scaleability  Issues  for  STOW 


Stuart  D.  Cheshire 
Stanford  University 

Comparison  of  ODF  and  Geographical  Grid  Multicasting 

I  saw  something  in  Dan  van  Hook’s  talk  that  no  one  else  seemed  to  stress,  perhaps 
even  Dan  van  Hook  himself  (but  that’s  okay — scholars  find  more  in  Shakespeare  than  he 
was  probably  ever  consciously  aware  of)- 

I  heard  several  times  that  “On-Demand  Forwarding’’  would  “probably”  reduce  traf¬ 
fic  better  than  geographical  multicast  groups  would,  but  it  was  not  stated  very  forcefully 
and  it  did  not  seem  to  evoke  any  conclusive  agreement.  I  would  say  that  ODF  would  d^- 
nitely  reduce  traffic  better  than  geographical  multicast  groups  would,  almost  by  definition, 
and  I  was  not  in  the  least  surprised  to  see  Dan  van  Hook’s  graph  showing  that  ODF  with 
170  multicast  groups  achieved  better  traffic  reduction  than  a  geographical  grid  with  678 
groups. 

If  I  understood  ODF  correctly,  then  it  allows  each  receiver  to  “determine  which 
entities  they  need  to  receive  state  firom.”  Surely,  if  each  receiver  is  receiving  precisely  the 
packets  it  needs  and  no  others,  then  this  is  the  optimum  solution?  However,  I  also  feel  that 
this  could  not  actually  be  made  to  work  because  each  receiver  does  not  have  perfect  knowl¬ 
edge  with  which  to  make  the  decision,  nor  unlimited  MIPS  with  which  to  do  the  calcula¬ 
tion,  nor  unlimited  bandwidth  for  control  information.  I  view  it  more  as  a  kind  of  DIS 
‘Turing  Machine” — something  one  would  never  build  but  a  model  against  which  to  com¬ 
pare  other  proposals. 

One  (undesirable)  way  we  might  actually  achieve  this  limit  could  be  with  an  omni¬ 
potent  gateway,  with  full  knowledge  of  the  DIS  application  semantics  and  full  models  of 
the  internal  states  of  all  the  participants.  It  could  arbitrarily  compress,  alter,  combine,  filter, 
and  discard  PDUs  to  send  precisely  the  information  needed  by  the  participants,  and  nothing 
more. 
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Multicast  is  a  powerful  general  purpose  mechanism,  but  how  do  we  know  how  well 
it  will  work  unless  we  already  have  some  notion  of  how  well  is  “well”  and  how  well  is  “not 
well”?  If  Dan  van  Hook  can  show  with  his  simulator  that  he  can  achieve  within  10%  of  our 
idealized  ODF  target  with  a  reasonable  number  of  multicast  groups  using  off-the-shelf  IP 
multicast  hardware,  then  I  would  be  prepared  to  call  that  a  win.  If  he  cannot,  then  perhaps 
we  will  have  to  consider  building  our  “smart”  gateways,  distasteful  as  that  may  be.  If  even 
our  idealized  ODF  model  does  not  produce  the  required  traffic  reduction,  then  we  have  to 
rethink  the  entire  approach.  In  any  eventuality,  1  think  it  is  valuable  for  us  to  know  which 
of  the  above  three  scenarios  we  are  facing. 

Encryption  Devices 

Discussion  was  between  using  Motorola  NES  (too  slow),  or  alternatively  abandon¬ 
ing  encryption  altogether  for  STOW  ‘94  (not  acceptable  to  NSA). 

A  third  possibility  not  mentioned  was  that  of  keeping  encryption,  but  lowering  the 
requirement  The  NES  costs  about  $15,000  and  has  a  few  hundred  kbps  throughput  at  best. 
Vendors  at  Interop  were  hawking  boxes  which  can  DES-encrypt  full  rate  ethemet  frames. 
One  company  I  saw  was  Semaphore  who  had  ethemet  to  T1  gateways  for  $7,(XX).  Its 
encryption  engine  can  do  up  to  6,0(X)  frames  per  second,  and  up  to  9Mbps— easily  enough 
to  keep  the  T1  link  completely  saturated  with  data. 

Perhaps  as  proof  of  concept  we  could  achieve  STOW  ‘94  using  DES,  and  leave  that 
component  of  the  system  upgradeable  to  a  Level  1  NSA  security  device  when  it  becomes 
available.  This  could  demonstrate  that  the  goals  of  the  STOW  project  are  in  fact  achievable, 
and  that  it  is  the  current  quality  of  encryption  devices  which  are  the  weakest  link  rather  than 
any  other  aspect  of  die  project 

ATM  Technolt^  Hopes  For  STOW  ‘97 

I  agree  with  David’s  and  Duncan’s  reservations  about  the  prevailing  blind  faith  in 
ATM.  I  do  not  think  that  ATM  is  going  to  come  riding  up  like  a  knight  in  shining  armor  and 
rescue  the  networking  industry.  I’m  not  old  enough  to  have  seen  a  great  deal  of  history  in 
the  computer  industry,  but  I  can  mention  diree  “data  points”  that  I  am  familiar  with 
(Table  A-1,  “Apple,  Ethernet,  and  ISDN,”  on  page  25). 

You  can  make  your  own  decision  about  whether  you  think  ATM  falls  into  the  Apple 
camp,  the  Xerox  PARC  camp  or  the  AT&T  camp,  but  personally  I’m  not  expecting  to  see 
anything  real  within  a  three-year  time  scale,  and  I  wouldn’t  bet  the  farm  on  it. 
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Table  A-1.  Apple,  Ethernet,  and  ISDN 


1.  Apple.  Computer  company. 

•  1990,  Apple  IIFX  costs  about  $8,000. 

•  1993,  Apple  Newton  gives  twice  the  MIPS  in  a  package  that 
weighs  under  a  pound,  fits  in  the  palm  of  your  hand,  runs  for  two 
weeks  on  alkaline  AAA  batteries  and  costs  about  $700. 

•  Time  scale:  3  years. 

2.  Ethernet.  Borderline  computing/communication  technology. 

•  1977,  Xerox  Ethernet  runs  at  10Mbps. 

•  1993,  first  faltering  steps  towards  a  100Mbps  Ethernet  standard. 

•  Time  scale:  16  years. 

3.  ISDN.  Telecommunications  technology. 

•  1963,  ISDN  announced. 

•  1993,  still  not  widely  available  even  in  the  USA,  never  mind  in 
less-developed  countries  in  the  world. 

•  Time  scale:  30  years. 

Dead  Entity  Server 

Useful  as  this  is,  I  would  resist  too  much  development  in  this  direction.  For  DIS  to 
be  a  success,  it  has  to  solve  the  dynamic  terrain  problem,  and  when  this  is  accomplished, 
the  facilities  promised  by  the  dead  entity  service  will  be  subsumed  into  the  more  general 
mechanism. 

If  the  dead  entity  server  has  to  be  built,  it  should  be  regarded  in  the  same  light  as 
the  NES  boxes,  i.e.,  as  a  temporary  solution  for  STOW  ‘94,  to  be  discarded  for  STOW  ‘97. 

Comments  on  Individual  Presentations 

Bill  Miller  (AT&T) 

His  lack  of  understanding  of  some  basic  principles  worried  me.  His  proposal  to  send 
only  incremental  changes  exposes  the  fact  that  he  had  not  even  considered  that  a  network 
of  this  scale  loses  packets.  If  it  did  not  lose  a  single  packet,  then  that  would  be  evidence  of 
gross  over-engineering  and  waste  of  money. 


He  was  obviously  too  used  to  thinking  in  terms  of  telephone  calls  where  you  have 
a  very  small  bandwidth,  only  two  endpoints,  and  charge  a  dollar  a  minute  to  pay  for  an  over- 
engineered  network.  I  do  not  know  how  much  AT&T  would  charge  to  set  up  a  100-way 
conference  call,  or  even  if  they  are  capable  of  doing  it.  1  am  fairly  certain  that  they  could 
not  set  up  a  1,000-way  or  10,000- way  conference  call. 

STOW  seems  so  far  outside  the  gamut  of  AT&T’s  experience  that  Bill  Miller  was 
not  even  able  to  answer  questions  with  plausible  sounding  guesses. 
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David  R.  Cheriton 


David  Cheriton  is  a  Professor  of  Computer  Science  at  Stanford  University,  Palo 
Alto,  California.  Professor  Cheriton ’s  primary  focus  of  research  has  been  on  high  perfor¬ 
mance  distributed  systems  with  considerable  emphasis  on  real-time  applications.  Addition¬ 
al  research  foci  include  issues  in  high-speed  communication,  multiprocessor  architecture, 
and  distributed  database  support.  His  research  group  developed  a  distributed  operating  sys¬ 
tem  called  V.  The  central  idea  in  the  V  system  is  to  provide  a  fast,  reliable,  and  secure  real¬ 
time  distributed  operating  system  kemal  on  which  real-time  applications,  distributed  data¬ 
bases,  and  ordinary  software  development/document  production  can  be  implemented  with 
maximal  performance  and  minimal  implementation  difficulty.  The  main  focus  has  been  on 
conununication  performance. 

Professor  Cheriton’s  research  has  been  largely  sponsored  by  the  Advanced 
Research  Projects  Agency  (ARPA),  and  he  plays  an  active  role  in  the  directions  of  this  com¬ 
munity.  In  particular,  he  was  a  member  of  the  ARPA  Gigabit  Working  Group,  a  task  force 
chairman  for  the  Distributed  Systems  Architecture  Board,  and  a  member  of  the  End-to-End 
Protocols  Task  Force  under  the  Internet  Advisory  Board.  In  this  role,  he  and  his  group 
developed  the  IP  multicast  extension  (IEEE  Request  for  Comments  (RFCs)  966,  988) 
which  is  now  a  draft  standard.  He  also  developed  the  VMTP  protocol  (IEEE  RFC  1045) 
which  is  considered  a  strong  candidate  as  a  next-generation  transpon  protocol  for  the  Inter¬ 
net,  addressing  issues  of  real  time,  security,  and  fault-tolerance.  Finally,  Professor  Cheriton 
has  been  an  Associate  Editor  of  the  ACM  Transactions  on  Computer  Systems  and  the  Dis¬ 
tributed  Conjuring  journal,  and  a  referee  for  all  the  major  publications  in  the  field.  Profes¬ 
sor  Cheriton  is  a  member  of  the  Institute  of  Electrical  and  Electronics  Engineers  (IEEE), 
Inc.,  and  the  Association  for  Computing  Machinery  (ACM). 

Professor  Cheriton  received  his  Ph.D.  in  computer  science  firom  the  University  of 
Waterloo,  Canada,  in  1978,  with  his  thesis  work  growing  out  of  the  development  of  the 
“Thoth”  portable  real-time  operating  system.*  This  system  was  used  in  a  number  of  real¬ 
time  applications  and  has  served  as  a  basis  for  several  other  commercial  real-time  systems. 
He  subsequently  spent  three  years  at  the  University  of  British  Columbia  in  Canada.  He  has 
been  at  Stanford  University  since  1981. 


*  D.R.  Cheriton,  M.A.  Malcolm,  L.S.  Melen,  G.R.  Sager,  “Thoth;  A  Portable  Real-Time  Operating  System,” 
Communications  of  the  ACM,  TIP.  (February  1979),  pp.  105-115. 
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Based  on  this  expertise.  Professor  Cheriton  has  worked  as  a  consultant  for  a  wide 
range  of  companies,  including  ESl/TRW,  Rockwell  International,  IBM,  Silicon  Graphics, 
Digital  Equipment  Corporation,  Institute  for  Defense  Analyses,  Texas  Instruments,  SRI, 
and  Dynamics  Research  Corporation.  He  regularly  gives  talks  and  lectures  on  distributed 
systems  research  at  the  major  universities  and  research  laboratories  throughout  the  United 
States. 


Dale  Henderson 

Dale  Henderson  is  employed  at  Los  Alamos  National  Laboratory,  Los  Alamos, 
New  Mexico.  He  joined  Los  Alamos  in  1966  upon  completing  his  Ph.D.  at  Cornell  Univer¬ 
sity.  After  four  years  in  experimental  plasma  physics,  he  moved  to  the  (then  new)  laser 
induced  fusion  program.  In  1975  Dr.  Henderson  became  the  leader  of  the  theory  group  in 
the  laser  induced  fusion  project  In  1979  he  moved  to  the  project  management  of  computer 
code  development  for  the  nuclear  weapons  design  program.  Soon  after  President  Reagan’s 
“Star  Wars”  speech,  Dr.  Henderson  recognized  the  need  for  a  flexible  comprehensive  sim¬ 
ulation  model  and  began  the  DETEC  (Defense  Technology  Evaluation  Code)  project  at  Los 
Alamos.  DETEC  was  adopted  as  the  major  software  vehicle  at  the  Strategic  Defense  Initia¬ 
tive  Oiganization’s  National  Test  Bed  (NTB)  in  1988.  Having  served  the  NTB  Joint  Pro¬ 
gram  Office  from  its  beginning.  Dr.  Henderson  undertook  a  FY  1990  assignment  to  the 
SDIO  as  Chief  Scientist  of  the  NTB. 

Duncan  C.  Miller 

Duncan  Miller  is  currently  a  Group  Leader  at  the  Massachusetts  Institute  of  Tech¬ 
nology  (MIT)  Lincoln  Laboratory  in  Lexington,  Massachusetts.  Dr.  Miller  has  been 
involved  with  man-in-the-loop  simulation  since  his  master’s  thesis  at  MIT’s  Man-Machine 
Systems  Laboratory  in  1964-65.  In  1983,  while  at  Bolt,  Beranek  and  Newman  (BBN),  Inc., 
he  formed  and  led  the  group  that  developed  the  original  Simulator  Network  (SIMNET) 
architecture  and  software.  He  has  been  actively  involved  in  the  extension  of  distributed 
interactive  simulation  systems  and  concepts  since  that  time.  He  is  a  member  of  the  Steering 
Committee  for  Distributed  Interactive  Simulation  Standards  (IEEE  Std  1278-1993)  and  the 
Defense  Science  Board  Task  Force  on  Simulation,  Readiness,  and  Prototyping.  He  previ¬ 
ously  served  on  the  Naval  Research  Advisory  Committee  Panel  on  the  Impact  of  Advanc¬ 
ing  Technology  on  Exercise  Reconstruction  and  Data  CoUection. 

Dr.  Miller  received  his  bachelor’s  (1965),  master’s  (1965),  engineer’s  (1967)  and 
doctorate  (1969)  degrees  in  mechanical  engineering  from  MIT.  His  principal  areas  of  study 
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included  control  theory,  human  operator  performance  modeling,  human  factors,  and  per¬ 
ceptual  psychology. 


David  L.  Mills 

David  Mills  is  a  Professor  of  Electrical  Engineering  at  the  University  of  Delaware, 
Dover.  He  currently  leads  projects  in  high-speed  networks  and  internetworking  research 
sponsored  by  ARPA  and  National  Science  Foundation  (NSF).  His  research  activities  have 
been  concentrated  in  the  areas  of  network  architecture,  protocol  engineering,  and  experi¬ 
mental  studies  using  the  Internet  system.  He  is  a  member  of  the  Internet  Research  Steering 
Group  and  formerly  chaired  the  Internet  Architecmre  Task  Force.  He  is  also  an  advisor  to 
the  NSF  and  was  principal  architect  of  the  NSFNET  Phase-I  Backbone  network. 

Before  joining  the  IDelaware  faculty  in  1986,  Dr.  Mills  was  a  Director  (Networks) 
at  M/A-COM  Government  Systems  Division  (Linkabit),  and  led  ARPA-sponsored  research 
and  development  projects  in  packet-switching  network  architectures  and  application  proto¬ 
cols.  Previously,  he  was  a  Senior  Research  Scientist  at  COMSAT  Laboratories  where  he 
worked  in  the  areas  of  packet-switching  satellite  and  internetworking  technologies,  and 
was  an  assistant  professor  of  computer  science  at  die  University  of  Maryland,  Takoma, 
Maryland,  where  he  worked  on  several  research  projects  in  distributed  computer  networks 
and  operating  systems. 

Dr.  Mills  earned  a  doctorate  in  computer  and  communication  sciences  at  the  Uni¬ 
versity  of  Michigan,  Ann  Arbor,  Maryland,  in  1971,  and  has  held  postdoctoral  positions  at 
the  University  of  Edinburgh,  Scotland,  and  the  U.S.  Defense  Communications  Agency 
[now  the  Defense  Informations  Systems  Agency].  He  has  published  and  lectured  extensive¬ 
ly  on  data  communications,  computer  networks,  and  operating  systems,  and  has  been  a  con¬ 
sultant  to  a  number  of  corporations  and  government  agencies.  He  is  a  member  of  Sigma  Xi, 
ACM,  and  the  IEEE  Computer  Society. 

Stuart  D.  Cheshire 

Stuart  Cheshire  is  a  fourth-year  Ph.  D.  student  at  Stanford  University,  Stanford,  Cal¬ 
ifornia,  studying  under  David  Cheriton  in  networks  and  distributed  systems.  Mr.  Cheshire's 
primary  focus  of  research  has  been  in  protocols  for  distributed  interactive  simulation.  The 
results  of  some  of  this  work  can  be  seen  publicly  in  the  form  of  the  popular  Internet  game 
“Bolo,”  a  networked  multi-user,  real-time  tank  battle  simulation. 


B-5 


His  research  has  been  sponsored  in  part  by  the  U.S.  Army  Research  Institute,  Alex¬ 
andria,  Virginia,  for  its  contribution  to  Army  research  into  team  training  methodologies. 
Mr.  Cheshire  received  his  First  Qass  Honours  B.A.  in  Computer  Science  from  Sidney  Sus¬ 
sex  College,  Cambridge,  England,  in  1989,  and  was  awarded  an  honorary  M.A.  in  July 
1993.  He  has  written  for  numerous  computer  magazines,  and  has  worked  in  the  computer 
industry  with  Madge  Networks,  a  company  manufacturing  IBM-compatible  Token  Ring 
network  hardware  and  software. 

Mr.  Cheshire  is  the  author  of  the  Stanford  print  accounting  package  which  authen¬ 
ticates  and  bills  students  for  netv'ork  printing  from  their  in-room  connections,  and  the  Stan¬ 
ford  Software  Librarian,  a  fault-tolerant  distributed  software  license  management  system. 
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Simulation  Tools  for  Developing  and  Evaluating  Networks  and  Algorithms  in 
Support  of  STOW  94 . C-3 

Daniel  J.  Van  Hook,  Loral  Advanced  Distributed  Simulation 

Intelligent  Gateway  for  Defense  Simulation  Internet . C-37 

William  D.  Miller,  AT&T  Bell  Laboratories 

STOW  94  Network  Requirements  &.  Evaluation  of  Solutions . C-73 

Joshua  Seeger,  BBN  Systems  &  Technologies 

Today’s  DSI . C-89 

Lou  Berger,  BBN  Systems  and  Technology 
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:LOG-TZMESTAMP  0 

:POU-SZZC  ISO 

:PDU  #S (ENTZTY-STATE 

: HEADER  «S (PDU-HEADER 

tVERSZON  1 
:EXERCZSE  1 
:KIND  1 
;UNUSED_8  0) 

:ENTZTY-ZD  #S (ENTZTY-ZD 

:SZMULATOR  #S (SZMOLATOR-ADDRESS 
:SZTE  1005 
:HOST  1) 

: ENTITY  26) 

:UNUSED_8B  0 
: FORCE- ID  2 

;ENTITY-TYPE  #S (ENTITY-TYPE 

; ENTITY-KIND  1 
•.DOMAIN  2 
: COUNTRY  164 
.•CATEGORY  2 
:  SUBCATEGORY  8 
•.SPECIFIC  0 
:EXTRA  0) 

;GUISE  is (ENTITY-TYPE 

: ENTITY-KIND  0 
: DOMAIN  0 
: COUNTRY  0 
: CATEGORY  0 
:SUBCU1TEG0RY  0 
: SPECIFIC  0 
:EXTRA  0) 

:TZMESTAMP  2378810926 
tUXyiTION  is  (NORLD-COORDINATES 

:X  -2635379. 0267692064d0 
:Y  -4397589. 30472663da 
:Z  3791781. 967626985d0) 

:VELOCITY  iS (LINEAR-VECTOR  :X  0.0  :Y  0.0  :Z  0.0) 

:  ORIENTATION  iS (EULER-ANGLES 

:PSI  3559208823 
: THETA  413347928 
:PHI  2669003701) 

:DEAD-RECKON-PARMS  iS (DEAD-RECKON-PARMS 

: ALGORITHM  2 
:UNUSED-8  0 
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:UNUSED-32-2  0 
:UNUSED-32-3  0 

: ACCELERATION  iS (LINEAR-VECTOR 
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:Y  0.0 
:Z  0.0) 

:ANGULAR-VELOCITY  tS  (ANGULAR-VELOCITY-VECTOR 

:ROLL  0 
:PITCH  0 
.'YAW  0)) 

: APPEARANCE  0 

.'MARKING  is  (ENTITY-MARKING  : CHARACTER-SET  0  .'TEXT  "MDTS") 
:CAPABILITIES  iS (ENTITY-CAPABILITIES 

: AMMUNITION-SUPPLY  0 
; FUEL-SUPPLY  0 
:MISC-SUPPLY  0 
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:  PARTS  «(#S (ARTICULATED-PART 
tCHANGE  lOei 
: PART-ID  31232 
:TyPE  4011 

rVALUE  #S (SIXTY-FOUR-BITS 

••WORDS  #(1357661873  0)))))) 
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